The interest in providing a depth sensation when displaying an image on a 2D display is growing rapidly, notably in virtual reality applications and computer games. Various forms of providing depth cues, such as influencing the brightness level or the size of an object, are known. Particularly, stereopsis, or stereoscopic vision, receives much attention as a technique for providing depth sensation. Stereopsis is evoked by presenting to a person two 2D images of the same scene as observed from two positions a little way apart. One of the images is presented to the left eye, the other one is presented to the right eye. The two images are parallactically related. The term “parallax” refers to the apparent displacement or the difference in apparent direction of an object as seen from two different points not on a straight line with the object. Parallax allows a person to perceive the depth of objects in a scenery.
Many display techniques for displaying stereoscopic images are known. Using a time-parallel technique, both output images are presented simultaneously to one or two displays. For example, both images may be filtered with complementary colors and superimposed on one display. The observer wears glasses with filters that match the projection filters. Alternatively, both images may be displayed alongside on one display and viewed using a viewer which directs each image to the correct eye. As another example, two displays may be used to present two differently polarized pictures, which are viewed through correspondingly polarized glasses. Alternatively, both images may be presented using a head-mounted device with separate displays for each eye. Also time-multiplexed techniques may be used, where the left and right images are alternatingly displayed on one display. As an example, one image is written to the even scan lines of a monitor and the other image to the odd scan lines. A shutter system is used to occlude the left eye when the right-eye image is displayed and to occlude the right eye when the left-eye image is displayed. The shutter system may be mounted in glasses worn by the observer. Alternatively, a shutter with a controllable polarizer is placed in front of the display and the observer wears a head-mounted device with polarized glasses.
A possible video format that is suited for rendering a 2D image from different viewpoints is a 2D video format enriched with depth information. Typically, the 2D input image is given in the form of an array of pixels. It may, for instance, have been obtained using a camera or computer graphics. For each of the pixels of the input image additional depth information is available or, similarly, for each point in the scene, a distance is given of the point to the camera (or to another reference point, line or plane, such as a projection screen). Such a format is usually referred to as a 2.5D video format. The depth information allows modeling of the original image as a set of samples of a flat image, but also as a set of samples of an image projected onto a terrain. FIG. 2A shows a cross section of the sampled terrain. From the camera position a ray is cast to each pixel of the screen. The lengths of the arrows along the rays indicate the depth values of the pixel samples, also referred to as texels (texture elements). The 2.5D video format represents a subset of the full 3D model of the world. Rendering from other viewpoints can be accomplished by projecting the terrain onto the image plane from the desired viewpoint. Based on the parallax when observing the same object from the visual point of the eyes and from the camera point, the pixels of the left eye image and the right eye image can be derived from the pixels of the input image by shifting the pixels. By choosing the horizontal axis of the coordinate system as being parallel to the line connecting both eyes, only a horizontal shift occurs. The amount of shift of a pixel depends on the depth information. FIG. 2B shows that after viewpoint transformation, the density of the projected input pixels is not uniform in the output domain. Hence, a resampling procedure is required. Existing video processing hardware/software designed for processing broadcast video signals is able to yield high-quality output images. Such hardware/software is able to scale an image to different display formats, such as 4:3 and 16:9 and perform resampling and band limiting of the signal for such a conversion. The video processing usually occurs per horizontal scan line of the image.